Non-Linear Feature Selection for Classification

This paper addresses the issues associated with performing feature or parameter selection for non-linear classifiers using a basis pursuit regularization framework. New results on representing the feature selection problem as a primal/dual calculation for both hard and soft margin classification problems are derived, and it is shown that optimal feature selection can be posed, in dual form, as a set of 2n linear inequality constraints. While this is efficient, it does limit the technique to non-linear kernels that have a finite expansion, such as polynomials. The issues associated with both efficiently calculating a polynomial basis pursuit classifier are then addressed and the technique is shown to improve discrimination performance on the MNIST digit set.